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Abstract 

We study continuous wavelet transforms associated to matrix dilation groups giving rise 
to an irreducible square-integrable quasi-regular representation on L 2 (K d ). It turns out that 
these representations are integrable as well, with respect to a wide variety of weights, thus 
allowing to consistently quantify wavelet coefficient decay via coorbit space norms. We 
then show that these spaces always admit an atomic decomposition in terms of bandlimited 
Schwartz wavelets. We exhibit spaces of Schwartz functions contained in all coorbit spaces, 
and dense in most of them. We also present an example showing that for a consistent 
definition of coorbit spaces, the irreducibility requirement cannot be easily dispensed with. 

We then address the question how to predict wavelet coefficient decay from vanishing 
moment assumptions. To this end, we introduce a new condition on the open dual orbit 
associated to a dilation group: If the orbit is temperately embedded, it is possible to derive 
rather general weighted L p,9 -estimates for the wavelet coefficients from vanishing moment 
conditions on the wavelet and the analyzed function. These estimates have various applica- 
tions: They provide very explicit admissibility conditions for wavelets and integrable vectors, 
as well as sufficient criteria for membership in coorbit spaces. As a further consequence, one 
obtains a transparent way of identifying elements of coorbit spaces with certain (cosets of) 
tempered distributions. 

We then show that, for every dilation group in dimension two, the associated dual orbit 
is temperately embedded. In particular, the general results derived in this paper apply to 
the shearlet group and its associated family of coorbit spaces, where they complement and 
generalize the known results. 

Keywords: square-integrable group representation; continuous wavelet transform; coorbit spaces; Ba- 
nach frames; vanishing moments; shearlets; temperately embedded orbits 
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1 Introduction 

One of the starting points of wavelet theory was the introduction of the continuous wavelet 
transform by Grossmann, Morlet and Paul [21| . Building upon the group-theoretic understand- 
ing of this transform, higher-dimensional analogues of the continuous wavelet transform were 
later introduced, starting with |25| . followed by (amongst others) [14]. Since then, higher- 
dimensional wavelet systems (some with, some without underlying group structure) have been 
studied extensively. However, while the representation-theoretic aspects of these transforms are 
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by now well-understood [2j [HI [24J, [T71 [18], the same cannot be said of their approximation- 
theoretic properties, with the notable exception of some specific examples such as the similitude 
groups or the shearlet group and its higher-dimensional relatives. It is the purpose of this pa- 
per to initiate a systematic study of the approximation-theoretic aspects of higher-dimensional 
continuous wavelet transforms, with particular emphasis on the role of the underlying group. 
To this end, the following question is addressed: Given a function / and an admissible dilation 
group H, what do we need to know about / and H to predict a certain decay of the wavelet 
coefficients? With the notable exception of the similitude group (where these questions are 
closely related to the theory of isotropic homogeneous Besov spaces) and the shearlet group 
(including some higher-dimensional relatives), this question has not been studied systematically 
in higher dimensions. 

Let us quickly introduce the basic notions of continuous wavelet transforms in dimension d > 1. 
For more details and background, we refer the reader to the cited papers, and to Section[2j We fix 
a closed matrix group H < GL(d,R), the so-called dilation group, and let G = R d x H, which 
can be viewed as the group of afiine mappings generated by H and all translations. Elements 
of G are denoted by pairs (x, h) 6 W 1 x H, and the product of two group elements is given by 
(x,h)(y,g) = (x + hy,hg). G acts unitarily on L 2 (IR rf ) by the quasi-regular representation 
defined by 

[7r(x,h)f](y) = \det(h)\- 1 / 2 f{h- 1 (y-x)) . (1) 

We assume that H is an admissible dilation group, which in the course of this paper means 
that 7r is an (irreducible) square-integrable representation. Square-integrability of the 
representation means that there exists at least one nonzero admissible vector tp G L 2 (R n ) 
such that the matrix coefficient 

(x,h) h-> {ip, tt(x, h)ip) 

is in L 2 (G), which is the L 2 -space associated to a left Haar measure on G. Recall that in this 
case the associated wavelet transform 

Vfy : L 2 (R d ) 3f^ ((x, h) ^ (/, vr(x, h)^)) 

is necessarily a scalar multiple of an isometry, which gives rise to the wavelet inversion for- 
mula 

f = — I W. 4) f(x,h)7r(x,h)d f i G (x,h) . (2) 
H Jg 

This formula also explains the interest in admissible vectors, which can be interpreted as basic 
building blocks with the property that every function in L 2 (K d ) has a continuous expansion in 
the wavelet system {tt(x, /i)V0(a;,/i)eG- 

As already mentioned, the L 2 -theoretic aspects of this type of decomposition are by now quite 
well understood, mostly thanks to L 2 -specific tools such as the Plancherel formula. Of particular 
importance for an understanding of the wavelet transform is the dual action, which is just the 
(right) linear action M d x H B (£, h) \-t /i T £: By the results of [HI [18], H is admissible iff the dual 
action has a single open orbit O = {h T ^Q : h £ H} C M. d of full measure (for some £o £ O), such 
that in addition the stabilizer group H^ = {h € H : h T ^o = £o} is compact. (This condition 
does of course not depend on £o £ O.) The dual open orbit will play a key role in this paper as 
well. 

Let me now sketch the contents of the paper. Section 2 puts the question of wavelet coefficient 
decay on a proper functional-analytic foundation, by showing that every admissible dilation 
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group gives rise to an integrable group representation (see Theorem 12. 1\ which is an extension 
of a result in [22J). This observation allows to invoke the powerful coorbit theory developed 
by Feichtinger and Grochenig |10l [TT\ [T2"l [20] , which provides a consistent and robust way of 
quantifying wavelet coefficient decay by imposing norms on the coefficients. Here we focus on 
the spaces Co\T v q of (Besov-type) coorbit spaces associated to weighted lf' q . A particular asset 
of the coorbit approach is the existence of discretizations and atomic decompositions, which are 
available once certain additional technical conditions (involving oscillation estimates) are met. 
In Lemma 12.7} we prove that these conditions hold true if the analyzing wavelet is a Schwartz 
function that is bandlimited to a compact subset of the dual open orbit O. Hence any such 
function can be used to define an atomic decomposition (Theorem 12. 9h . simultaneously valid in 
all coorbit spaces. 

While these observations are interesting (and somewhat surprising in their generality), they do 
not answer the question how to predict wavelet coefficient decay in a fully satisfactory manner, 
at least for two reasons: The first one is the obvious objection that the coorbit definition 
only provides a circular answer to the question why a function has good coefficient decay. In 
other words, coorbit theory allows to identify functions that have good decay, in a functional- 
analytically sound and consistent way, but it does not a priori help characterize them. A 
somewhat more subtle point is that by definition, elements of coorbit space are contained in 
the dual H\ w of a certain Banach space of functions on G. Thus it would be nice to have a 
means of identifying the elements of coorbit spaces with functions (or, more generally, tempered 
distributions) on M. d rather than viewing them as elements of a fairly abstract dual space. 
It is the purpose of the remaining sections of the paper to provide more concrete answers. For 
this purpose, we generalize one concept which may considered as one of the cornerstones of 
wavelet analysis in one dimension, namely the simple observation that vanishing moments of a 
wavelet, together with (local) smoothness of the analyzed signal, results in wavelet coefficient 
decay. For general dilation groups H, the correct generalization of this notion turns out to be 
that of vanishing moments in O c , (see Definition I3.2p . where O c denotes the complement of the 
dual orbit. Roughly speaking, the points in of O c can be understood as the "blind spots" of the 
wavelet transform; i.e., they correspond to frequency information that the wavelet transform 
typically cannot resolve very well. Thus intuition suggests that functions that have little or 
no frequency content in those areas (or nearby those areas) can be well approximated, and it 
is the purpose of Section [3] to make this intuition work. It turns out that under additional 
technical assumptions (formulated in the notion temperately embedded dual orbit, see Definition 
13. ip . it is indeed possible to predict membership in a coorbit space from suitable smoothness 
and vanishing moment assumptions (Theorems I3.6|4L4"|) . Moreover, the same arguments allow 
to easily identify integrable vectors that qualify as analyzing wavelets for coorbit spaces. In 
particular, we show that there exist such vectors in C£°(R d ) (Theorem l4.3p . Finally, we introduce 
a natural closed subspace J r ~ 1 S(0) of Schwartz functions that is continuously, and for p, q < oo 
densely, embedded in CoLf,' 9 (G). A useful byproduct of the embedding is the dual embedding 
allowing to identify elements of coorbit spaces with (cosets of) tempered distributions. 
In the final section of the paper, we verify that for all admissible dilation groups in dimension 
two, the associated open dual orbits are temperately embedded, thus making the results of the 
previous sections available for all wavelets in dimension two. For the shearlet case, these results 
considerably extend the previously derived results in [6l [U [9] . 
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2 Coorbit spaces associated to admissible dilation groups 

In this section, we introduce the necessary notations, and comment on the relationship of in- 
tegrability versus square-integrability for quasi-regular representations. In the following, we 
simply write j -dx for integration against left Haar measure. The left Haar measure of G 
is then expressed as d(x,h) = \ a\et(h)\~ l dxdh, and the modular function of G is given by 
Ag(x, h) = A#(/i)| det(/i)| -1 . Given / € L 1 (R rf ), its Fourier transform is defined as 

H0(0 == f(0 ■= I f(x)e- 2 ^dx, 



with (•, ■) denoting the euclidean scalar product on M. d . We will use the same symbol J- for 
the Fourier transform of tempered dsitributions. For any subspace X C S'(M. d ), we let J r ~ 1 X 
denote its inverse image under the Fourier transform. 

In order to avoid cluttered notation, we will occasionally use the symbol X < Y between 
expressions X, Y involving one or more functions or vectors in M. d if there exists a constant 
C > 0, independent of the functions and vectors occurring in X and Y, such that X < CY. We 
fix a norm • : R d — > R. The precise choice of norm is immaterial in the following arguments, 
as it only affects the constants. (The examples in Section [5] illustrate that it may be convenient 
to be able to switch between norms.) We also fix a norm || • || : W dxd — > M + . Again, the 
precise choice is immaterial. Note that since all norms on finite-dimensional vector spaces are 
equivalent, we have the submultiplicativity properties \\gh\\ ^ \\g\\ \\h\\, for all matrices g,h, and 
similarly \g£\ ^ \\g\\ |£| for all matrices g and vectors £. We will (somewhat inconsistently) 
always use \a\ = Y2i=i a i f° r multiindices a, since in the following no serious confusion can arise 
between vectors and multiindices. 

Recall that a weight on a locally compact group G is by definition a continuous function w : G — >• 
R + satisfying the submultiplicativity condition w(xy) < w{x)w{y). Throughout this paper, we 
assume to have fixed s 6 M + , and weights v : G — > R + and w : H — >■ M + such that for all (x, h) G 
G: v{x, h) < (1+ \x\) s w(h). Here | • | : R rf — > M. is an (arbitrary) norm. Examples of such weights 
can be constructed by fixing a weight w on H, and letting v(x, h) = (1 + | a; j ) s ( 1 + ||/i||) s t(;(/i), 
where \\h\\ denotes the operator norm of the linear mapping (M. d , \ ■ |) — > (M rf , | • |) given by matrix 
multiplication with h. Note that as a result of submultiplicativity, every weight v on G fulfills 

v(x, h) < v(x, l)v(0, h) , 

and the restrictions to the subgroups M. d x {1} and {0} x H are again weights. 
We define, for 1 < p, q < oo, 



L^(G) = \f:G^C : f ( f 



\ q/p 

\F(x, h)\ p v(x, h) p dx J dh< oo 



with the obvious norm, and the usual conventions regarding identification of a.e. equal functions. 
We write h%{G) = L P ' P (G). The corresponding spaces for p = oo and/or q = oo are defined by 
replacing integrals with essential suprema. We will also use 

L p (R d ) = U Borel-measurable : J \f(x)\ p (l + \x\) sp dx < ooj . 
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Of particular interest is the space of u-integrable vectors, defined as 

Hi, v = {/ G L 2 (R d ) : W f f G Ll(G)} . 

For r, m > 0, we let 

|/| r>m = sup (l + \x\)- m \d a f(x)\ . 

x£R d ,\a<r 

denote the associated Schwartz norm of a function / : R d — > C with suitably many partial 
derivatives. 

The following result is known for the case of trivial dual fixed groups and weight w(x,h) = 
Ag (/i), see [22 1 . The general case needs only little adjustments. In the following C£°(0) 
denotes the space of / E C£°(R d ) such that supp(/) = /^(CX {0}) C O. 

Theorem 2.1. Assume that ir is square-integrable. Then it is v-integrable. More precisely: If 
t/j G T~ x C c {0), then G L\(G). 

Proof. Let ip be as in the theorem. We first rewrite the wavelet transform as 
W^(x) = det(/i)" 1/2 (if) * i/>*) (x) = det(h)- 1/2 ■ D h ^ (x) 

where ip*(x) = ip(—x), and Dhifj(^) = ip(h T £). Thus we can use our assumption on the weight 
v to estimate 

PWH14 < / 
Ju 

Since the transposed action of H on R d is smooth, the vector-valued mapping 

h^$-D h $e C c {0) 

is continuous with respect to the Schwartz topology. Since the Fourier transform is an automor- 
phism of S(R d ), and S(R d ) C Ll(R d ) continuously, it follows that the mapping 

h 1— > || £>^) v || L iu;(/i)| det(/i)r 1/2 

is continuous as well. It is also compactly supported: Fixing any £0 G O, let p^ : H — > 0,h i-> 
h T £o. Then p^ is continuous, and the induced canonical bijection between quotient H^ \ H 
and orbit O is a homeomorphism. Here H^ denotes the stabilizer of £oi which is compact by 
assumption. But this implies that K = pT^supp^)) is compact. Now tp ■ Dhip ^ implies the 
existence of hi E K such that h\h G K as well, thus finally h £ K~ 1 K, which is compact. 
Since the Haar measure of the compact set K~ X K is finite, it follows that HW^IIl 1 < 00. □ 

Now general coorbit spaces, with respect to suitable Banach function spaces Y on G, are defined 
as follows [10]: It can be shown that Hi, v , when endowed with the obvious norm, is a Banach 
space, which is invariant under tt. Denote by r H.\ v the conjugate dual of Hi >v . Thus Hi )V C 
L 2 (R d ) C Wi v , and the sesquilinear map L 2 (M d ) x %\ iV 9 (/, g) h-> (/, g) can be uniquely 
extended to Hi v X %i tV . Hence, if we fix a nonzero element ifi G Hi )V , we can define the 
continuous wavelet transform of / G Hi v by the same formula as for elements of L 2 (R d ). 



w(h)\det(h)\-^ 2 dh 
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Now let Y be a Banach function space on G. We assume that Y fulfills the conditions of [2Q|, 
2.2], i.e. it is a Banach function space continuously embedded in L* (G), and fulfilling certain 
compatibility conditions with convolution. We remark that the spaces Ly' q (G), which are our 
main concern, all fall in this category. The following definition spells out the compatibility 
condition: 

Definition 2.2. A weight vq is called control weight for Y if it satisfies 

v (x,h) = A G (x,hy 1 v ((x,hy 1 ) , 

as well as 

max (HL^ii ||y_>y, \\R( x ,h) \\y->Y, H-R^-i ||y^yA G (x, h)' 1 ) < v (x, h) 
where Li x ^\, Ri x ,h) are ^ e ft an d right translation by (x,h) G G. 

Now, if Vq is a control weight for Y, with nontrivial Wi t v , the coorbit space associated to Y is 
defined by picking a nonzero ip G Hi^o and then letting 

Co(F) = {/ G H~ tV0 : HV G Y} 

with the norm ||/||co(y) = ||VV^)/||y. A cornerstone of coorbit theory is the fact that the space 
CoY thus defined is independent of the analyzing vector, as well as of the precise choice of 
controlling weight; see [llj for details. 

The following lemma notes that for weighted L p,IJ -spaces of the type considered here, there exists 
a control weight v$ with vo(x, h) < (1 + \x\) s wo(h) as well, for a suitable continuous weight wq. 
Hence all results in this paper which state weighted integrability properties of certain matrix 
coefficients can be applied to decide membership in coorbit spaces Co(Y), but also to decide 
whether a wavelet is a suitable analyzing wavelet for Co(Y) (we just have to switch the weight). 
For these reasons we will not explicitly distinguish in these results between v and vq. 

Lemma 2.3. Let v : G IR + denote a weight with v(x,h) < (1 + \x\) s w(h). Then there exists 
a weight wq on H and a controlling weight vq on G for (G) with 

v (x,h) < (1 + \x\) s w (h) . 

Proof. Let 

^iC^,^) = (1 + + + + + ||/ i - 1 ||) s (l + ^ G (0,/ i ))(l + ^ G (0,/z)- 1 ). 

First note that (x, h) \— >■ (1+ + \\h\\) s is indeed a weight on G, as are the remaining factors 

in the definition of v±. Thus v\ is a weight on G. We next prove that v\ controls the translation 
norms. We first consider right translation and the case q < oo. Given F G Ly' q (G), we use the 
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invariance properties of the various measures and functions to compute 

\«/* dg 



\R{x,h)F\\Y 
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\F((y,g)(x,h)\v(y,g) p dy) 
\F iy + gx , gh) \ Pvi y, 9 y d y 
\F(y,gh)\Pv(y-gx, g ydy\ 



|det(<?) 



g/p 



dg 



\det(g) 

\ q/p dg 
\det{g) 

ilv 



\F(y,g)\Pv(y-gh-'x,gh- r ydy I A H (h) 



dg 



< 

< 
< 
< 
< 



F 



=v((y,g)(x,h)-i) 

\F(y,g)\Pv(y,g)Pv(-h~ 1 x,h- 1 )dy j 

F\\ q Y A G (x, hy^i-h^x, h- v ) q 
F\\ Y A G (x, h)- x {l + \h- x x\) qa w(h) q 
F||«-(l + A G (0,/ l )- 1 )(l + |x|r(l + ||/ l - 1 | 

V1 (x,hy . 



\ i/p 



A H (h) 



\det(gh-i) 
dg 

\det(gh-i) 



(l + w(h)) q 



\1 



The other cases are dealt with analogously. Thus the symmetrization vq of vi, defined by 

v (x,h) = vi(x,h) + A G (x,hy 1 v 1 ((x,hy 1 ) 
is a controlling weight. Furthermore, the calculation 

v^ix^h)- 1 ) < (i + i^xDXo,^- 1 ) < (i + IxD^i + h^idxo,^- 1 ) , 

together with the above estimates, shows that the symmetrization vq also obeys an estimate 
vo(x,h) < (1 + \x\) s wo(h), with a suitable weight wo on H. □ 

Remark 2.4. Since one is often interested in using a single wavelet to characterize whole classes 
of coorbit spaces, it is worthwhile to study the dependence of the controlling weight on the space 
Y . Fixing so > and a weight w on H , the Lemma shows that one can choose a controlling 
weight vq working simultaneously for all 1 < p, q < oo and all < s < sq. Thus a wavelet in 
T-Ll Q can be employed for the characterization of all coorbit spaces Co(Y) with Y = lX ,q (G). 

It is fairly intuitive that the isotropic homogeneous Besov spaces can be understood as coorbit 
spaces associated to the action of the similitude group H = M + x SO(d) as dilation group: As 
soon as one chooses an isotropic window (such as the phi- functions of Frazier and Jawerth [13j ) , 
the continuous wavelet transform is constant on the S'0(d)-cosets. Hence the rotation group 
can be omitted altogether. But then we are left with the continuous wavelet transform with 
respect to scalar dilations, and the (^-transform characterization of [13J allows to understand 
these spaces as coorbit spaces, either with respect to the irreducible representation arising from 
the similitude group, or with respect to the reducible representation arising from the scalar 
action. In a similar way, the anisotropic Besov space studied in [3] can be read as coorbit spaces 
associated to cyclic or one-parameter dilation groups. Note however that in these cases the 
quasi-regular representation is reducible. 
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An admissible dilation group for which the associated coorbit spaces have been studied exten- 
sively is the shearlet group, see [3 El El E] and the references therein. We point out however 
that typically, attention there is restricted to the spaces Co(L^' p ), and only for weights v of the 
type v(x,h) = w(h). 

We will next establish atomic decompositions for the coorbit spaces. For this purpose, we need 
the notion of oscillation: 

Definition 2.5. Let U C G denote a neighborhood of the identity, and F : G — > C any function. 
We let 

oscu(F)(x) = sup{\F(x) - F(xy)\ :yeU}. 

The following lemma will help to effectively deal with the determinant factor in estimating the 
oscillation of the wavelet transform: 

Lemma 2.6. Let G be an arbitrary locally compact group, and v a weight on G. Let F G h~(G), 
and tp : G —> M + a continuous homomorphism. Let U be a neighborhood of the identity in G. If 
both F and oscuF are in L„(G), for a suitable weight v, then oscu(F ■ if) G L*- (G). 

Proof. We first note that 

oscu(F ■ ip)(x) <<p(x) supy?(y) obcuF(x) + ip(x) sup |1 - (p(y)\ \F(x)\ , (3) 

\yeU J \y£U J 

which follows from 

\F(x)ip(x) - F(xy)<p(xy)\ < ip(xy)\F(x) - F(xy)\ + ip(x)\l - ip(y)\\F(x)\ 

Our assumptions on F now yield that the right-hand side of ([3]) is in L^, (G) □ 

The next lemma paves the way towards existence of atomic decompositions. 

Lemma 2.7. Assume that it is square-integrable. If ip € J-~ l C c {0), there exists a neighborhood 
U C G of the identity such that oscr/(W^,V) £ -^i(C)- 

Proof. Throughout the proof, | • | : W 1 — > R + denotes the euclidean norm, and |j • || : M. dxd — > M + 
denotes the operator norm with respect to | • |. Let B r (x) C M. d denote the open ball with radius 
r and center x. We write W^(x,h) = |det(/i)| 1//2 W^V( 2; ; h), and v'(x,h) = \det(h)\~ 1 ^ 2 v(x,h). 
By Lemma I2U1 we need to show that osct/(W^,V') e L^, (G) 

We fix V = B\{x) and W = {h G H : \\h — id|| < 1/2}, and m > d + 1 + s, where s is the 
exponent of the translation part of the weight v. Then U = VxWcGis& neighborhood of 
the identity in G, and we will show finiteness of ||osc[/(W^,V')IIl 1 ■ 

The proof proceeds in several steps. We first show the existence of a continuous function 
ip : H x H — >• R + , dependent on ip, such that 



Vs, h e HVx G R d : sup 



W^,ip(x + hy,g) -W^ip(x,g) < <p(h, g) (1 + max(0, |a?| - \\h\\)) m 

(4) 
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For this purpose, we employ the mean value theorem, and estimate for y £ V 
Wipi/j{x + hy, g) - W^(x, g) 



< \hy\ V sup \d a (i;*(7T(0,g)r))(z)\ 

z£x+hV 



< 
< 



\a\<l 

\il>*(n(0,g)ij>*)\i,m, sup (l + \z\y 

zGx+hV 



,\\ |V'*(7r(0 )5 )^*)|i )m (l + max(0 ) |x| - 

where we used that V is the unit ball. The proof of dH) is finished when we prove that 

<p{h,g) = \\h\\ |V*(7r(0,^*)| lim 

defines a continuous function on H. Here only continuity of the last factor may not be obvious. 
It follows however from the continuity of H 3 g i— > 7r(0, g)ip* £ <S(R d ), continuity of convolution 
on and continuity of the Schwartz norms. 

We use this estimate to prove the existence of a constant C > 0, dependent on ip, such that 



sup 

(y,g)eu 



W^(x,hg) -W^(x + hy,hg) (1 + \x\) s dx < C sup <p(h, hg)(l + \\h\\) 1+m+s (5) 

g&W 



Plugging in (jl]) yields 



sup 



[1 + \x\Ydx < 



Wipip(x, hg) - W^(x + hy, hg) 

< sup (p(h,hg) (1 + max(0, \x\ - \\h\\))~ m (1 + \x)) s dx . 
new jR d 



(6) 



Now 



/ 



(1 + max(0, \x\ - \\h\\)) rn {l + \x)) s dx 
/ (1 + \x\) s dx+ / (1 + \x 

J\x\<\\h\\ J\x\>\\h\\ 



<\x\ 

< CUl + llh 



+ / (1 + I \x\ 



l (l + \x\) s dx 



'(1 + \x\) s dx 



where C\ denotes the volume of the unit ball. Now the estimate (1 + | |x| 
(1 + |x|) yields 



n + \\h\\) > 



[ (i+i \x\-\\h\\ \)- m (i + \x\ydx< (i + \\h\\r [ (i + 



x\) m - s dx 



which in summary yields 

© < sup <p(h,hg)(C 1 (l + 
g&W 



■ C 2 



HI + C 2 (l + \\h\\) m ) < C sup ip(h, hg)(l + \\h\\) m+s+1 

g&W 
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and ([5]) is proved. 
We next prove that 

V/i € HVg e WVy e R d : ^((hg^y) - ^hr x y)\ < \\^\i, m+ i (l + |^~ 1 2/|/2) 



(7) 



Indeed, by the mean value theorem, and using the definition of W: 
U{{hgr l y)-^{h- l y)\ < \{hg)- l y - h^y\ Mi, m +l 



1 



sup 

g— 1 h~ 1 y— h~ ^y\\ 
—771—1 



[l + \z\ 



-m—1 



< - IMhm+ilh^yl (l + \h- l y\/2) 

< \ 1^1/2) " m 

Here we employed the estimate \g~ 1 h~ 1 y — h~ 1 y\ < \h y\/2. 

We apply this observation to derive, with a positive constant C > 0: 



sup 



W^ip(x,hg) -W^ip{x,h) (1 + \x\) s dx < C max(l, \\h\\ m ) (8) 
For the proof of ([8]), we use the definition of the wavelet transform, as well as ([7]), to obtain 



/ 



sup 

g€W 



W^(x,hg) -W^(x,h) (l + \x\) s dx 



sup 

: g&W 



^{x + y) (*p{(hg) 1 y)-ip(h 1 y))dy 



'1 + \x\ 



[ [ \Tp(x + y)\hi>\ ljm+1 (l + Ih-iyW) m dy(l + \x\) s dx 

JR d JR d 1 



< 



*K h )(x)(l + \x\) s dx 

= II M * K hhi < IIV'IIliII^IIli , 

where we employed Kh(y) = (l + \h~ 1 y\/2) m , as well as the weighted version of Young's 
inequality, which holds because of the submultiplicativity of the weight x i-> (1 + |x|) s . Using 



h 1 y\ > \\h\\ 1 \y\, we can estimate 



i-i 



\K, 



h\\IA 



= / (l + lh-^D^il + lylYdy 

JR d 

< I max(l,||/i|r)(l + |y|) s - m d 2 /<C / max(l,||/i| 
jR d 

We can now finish the proof: We consider the function 

$:H3h^ / oscu(yVfip)(x,h)(l + \x\) s dx . 



By our assumption on the weight v, we have 

oscu(W^) 1 < f $(h)\det(h)\- 1/2 w(h)dh 
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First observe that $ is compactly supported: With the compact set K from the proof of Theorem 
12.11 we find that oscu(W^)(x, h) ^ for some x € M rf requires that either h € K^ 1 K, or 
hg £ K K for at least one g € U. In any case, this can only happen for h € K~ l KU~ l , which 
is relatively compact. Furthermore, the triangle inequality yields the estimate 

oscu(Wipip)(x,h) = sup Wfip{x + hy, hg) - W^{x,g) 

yeV,geW 



< sup 

y£V,g£W 



W^ip(x + hy, hg) - W^tp(x, hg) + sup W^tp(x, hg) - W^tp(x, g) 



Now ([5]) and ([8]) provide us with 

$0) < C sup <p(h, hg)(l + ||/i||) 1+m+s + C"max(l, \\h\\ m ) , 

which is bounded on K~^KU~ X . Since in addition w|det| -1 / 2 is bounded on compact sets, the 
proof is finished. □ 

The lemma implies together with dominated convergence that 

HosciKVMOIIlj-^O (9) 
as U runs through a neighborhood base of the identity. 

As a consequence, we obtain the existence of atomic decompositions. First we need to define 
?7-denseness and [7-separatedness. 

Definition 2.8. Let U C G denote a neighborhood of the identity, and Z = (^i)ie/ C G. 

(a) The family (zi)n=i is called [/-dense, if{J ieI ZiU = G. 

(b) The family (^)ig/ is called [/-separated, if ZiU n ZjU = 0, whenever i ^ j. It is called 
separated, if there exists a neighborhood U of unity such that it is U -separated. It is 
called relatively separated if it is the finite union of separated families. 

Note that [/-dense, relatively separated families always exist, for every neighborhood U of the 
identity. 

We can now apply the general machinery of [20J to derive the existence of atomic decompositions. 
As a further ingredient for this, we require a Banach sequence space Yd(Z), which is associated 
to the Banach function space Y according to \20\ 2.3]. It is natural to expect that for case 
Y = Lw q (G), the coefficient space Yd(Z) will be of the form i^j, for a suitable discrete weight 
Wd- The clarification of this question, together with the construction of compactly supported 
atoms, will be the subject of future work. The following theorem combines |20|. Theorem S] and 
[201 Theorem T], which are applicable because of ([9]). 

Theorem 2.9. Let ip G F~ l C™(0) \ {0}, let Y = L% q (G) with 1 < p, q < oo. Then there exists 
a neighorhood U C G of unity such that for all U -dense, relatively separated families (zi)i^i C G, 
the following statements are true: 

(a) There is a linear bounded map C : CoY — > Y^(Z) with the property that, for all f € CoY , 

/ = ^c(/)(^(^ , 

iei 

with unconditional convergence in \\ ■ \\c6Y- 
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(b) Conversely, for every sequence (c(zi))i£i £ Yd(Z), the sum 

converges unconditionally in \\ ■ \\coY, with \\g\\coY ^ ||(c(2i))i e j||y d (x). 

(c) The norms \\f\\coY and \\(yVi[>f(zi))iei\\Y d (z) are equivalent. Moreover, f G CoY iff 
(Wi,f(zi))ieieY d (Z). 

Remark 2.10. The neighborhood U in Theorem \2. 9\) is chosen according to the requirement 
that 

This shows that the same sampling density works for all coorbit spaces Y for which vo is con- 
trolling. Thus, for the same range of weighted L p ' q -spaces that we found in Remark \2.4\ it is 
possible to choose a wavelet ip and a sampling set Z such that the associated wavelet system is 
a Banach frame in those spaces. 

Repeating the proof of Theorem l2.lt with the obvious modifications, allows to identify a space 
of Schwartz functions contained in all coorbit spaces. The density statement is a consequence 
of Part (a) of Theorem 12.91 

Corollary 2.11. Let Y = YPJ{G) with l<p,q<oo. Then F^C^ip) C CoY. The inclusion 
is dense for p, q < oo. 

2.1 Window dependence of coorbit spaces in the reducible case 

As already mentioned, one of the key properties provided by coorbit space theory is consistency: 
The definition of Co(Y) is independent of the choice of the analyzing wavelet ip G This 
property ensures is that "nice" behaviour of the wavelet coefficients Ww,/ of a function / (as 
quantified by the coorbit space norm) can be attributed to the analyzed function /, rather than 
to the analyzing wavelet ip. It turns out that this property crucially depends on irreducibility 
of the underlying representation. While the results of [10\ [Tl] [T2] reflect this, and are only 
formulated for irreducible representations, it is not always easy for the reader to pinpoint where 
precisely the requirement is needed. Moreover, recent investigations of wavelet inversion formulae 
(such as [HI EH |T7] ) and of coorbit theory [3JE1H] beyond the irreducible setting indicate that 
irreducibility may not be essential for the development of coorbit space theory. For instance, the 
authors of [5] emphasize the property that the reproducing kernel associated with the wavelet 
inversion formula is integrable, as crucial for their approach, rather than irreducibility 
This subsection gives an explicit example of a reducible representation with associated integrable 
reproducing kernels, for which consistency is lost. This result is of some independent interest, 
as the above-mentioned sources from the literature indicate: If one discards the irreducibility 
condition, more care and/or additional considerations are required to justify a particular choice 
of wavelet and the definition of the associated function spaces. In the context of this paper, the 
main purpose of the example is to explain why we focus on irreducible representations. 
We consider the group G = M? x M + , with the multiplicative group acting diagonally, and let ir 
denote the quasi-regular representation of G acting on L 2 (M 2 ). We will construct two functions 
f,g£ L 2 (G) with the following properties: 
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(i) The wavelet transforms Wf,W g : L 2 (R 2 ) -> L 2 (G) are both well-defined and isometric. 
Hence / and g give rise to wavelet reconstruction formulae, and the image spaces of W/, W g 
have reproducing kernels. 

(ii) The reproducing kernels associated to each analyzing wavelet are integrable. More pre- 
cisely, W g g = Wff € L„(G), for any continuous weight of the form v(x, s) = vo(h). 

(iii) Wfg L*(G). In particular, the coorbit spaces Co(Lj(G)) defined using either W/ or W g 
as analyzing wavelets do not coincide. 



For this purpose, pick ^ ip £ C£°(M + ), satisfying 

\Ms)\ 2 



ds = 1 . 

'o * 

Then, define / € L 2 (M 2 ) via /(f) = Vo(|f I)- In addition, define 5 € L 2 (M 2 ) by 

?(f) = sign((f,(l,0) T }) ./(f). 
Now by construction of / and g, one has 



and this implies (i), by |19t Theorem 1]. 
The computation 

W f f(x,s) = \s\(f-(D s 7)) V (x) 



and the fact that / 6 C^°(IR 2 ) imply that indeed W// € L*(G), by the same reasoning as in 
the proof of Theorem 12.11 On the other hand, the fact that sign((f, (1,0) T )) is constant on all 
half- lines (i.e., dual orbits), implies that 



W g g(x,s) = \s\ (g- (D^ (x) = \s\ (/■ (z) = W f f(x,s) . 



Thus (ii) is established. 

Finally, the same calculations yield 



W f g(x,s) = \s\ (gDjy (x) = \s\ (sign«-, (l,0) r )) • / • (D s -!/)) V (i) . 

Now, whenever the function f 1— > sign((x, (1,0) T )) • /(f) • (Z) s /(f)) is not identically zero, it is 
discontinuous near the y-axis, in a way that cannot be remedied by changing the function on a 
set of measure zero. This yields that W f g(-, s) L X (]R 2 ), for all sEl + with W f g(-, s) ^ 0. By 
property (i) (and because g 7^ 0), this set has positive measure. But then Wjg L*(G). 
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3 Temperately embedded dual orbits and vanishing moments 

This section contains the central estimates of this paper. We start out by defining the notion of 
temperately embedded dual orbits. Given any point £ € O, let dist(£, O c ) denote the minimal 
distance of £ to O. Furthermore, by £' the point on O c with minimal distance to £, we define 
the auxiliary function 

d (0 = 1 + min{|A£ + (1 - A)£'| : < A < 1} . 
We then define A : O -> R+, 

(10) 

We let /i~ T the transpose inverse of h £ H 

Definition 3.1. Xei w : H ^ M + denote a weight function, s > 0, and 1 < g < oo. (9 is called 
(s, u>)-temperately embedded (with index I 6 N) i/ i/te following two conditions hold, 
for a fixed £o £ O: 

(i) The function H3h^ |det(/t)| 1/2_1/<? (l + \\h\\) s+d+1 w(h)A(h T £ ) e is in L«(fl"). 

(ii) The function H B h |det(/i)|- 1 / 2 - 1 / (? (l + \\h\\) s+d+1 w(h)A(h- T ^ is in L^H). 

If O is (s,q,w) -temperately embedded for all 1 < q < oo and s > 0, (with an index possibly 
depending on s and q), the orbit O is called ^-temperately embedded. 

Submultiplicativity of w, |det| and || • || imply that the conditions (i) and (ii) are independent 
of the choice of £o £ O. Furthermore, an obvious but useful observation is that whenever the 
weight w' is dominated by the weight w, then every ^-temperately embedded dual orbit is 
also w'-temperately embedded. Finally, note that since A < 1, it is possible to first achieve 
finiteness for each integral separately, and then simultaneously. The aim of this section is to 
derive estimates of the form 

I|WW||ip> 9 (G) ^ \f\s,m\^\s,m , 
under suitable vanishing moment conditions on / and ip, which we clarify in the next definition: 

Definition 3.2. Let r G N be given, f G L 1 (R <i ) has vanishing moments in O c of order r 

if all distributional derivatives d a f with \a\ < r are continuous functions, identically vanishing 
on O c . 

We will need several lemmas. The first one motivates the introduction of our auxiliary function 
A from above. 

Lemma 3.3. Assume that f has vanishing moments in O c of order r, and that m > r. Then 
there exists a constant C > 0, dependent only on r, such that for all £ € R. d : 

no <c\f\ r , m A(o r 
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Proof. Let £ € O be arbitrary, and £' € O c the point closest to £ used to define do(£)- Let 
: i i—)- /(£' + and consider the Taylor expansion of 99 of order r — 1 around 0. By 
assumption, the Taylor polynomial is zero, and the remainder term is estimated by the chain 
rule and the assumption on £' by 



t>\ r 



r! " U "'"V!(1 + |£' 

since m > r. This proves the lemma. □ 
In the following lemma, we use the convention d/00 = 0. 

Lemma 3.4. Let 1 < p < 00. Pick an integer t > wif/i t > s + cZ/p. T/ien there exists a 
constant C > (WZy dependent on p and s), such that, for all g € L 1 (M d ) ; 

\\g\\v < C max ||<9 a g||i . 
|a|<t 

Proof. The estimate is trivial if one of the cPg is not integrable. If all of these functions are 
integrable, we obtain that T~ 1 (d a g)(x) = {2mx) a g{x), and g is bounded. This implies 

\g(x)\ r< (1 + Mr* max ||<9 a ?||i . 

I«l<* 

and the estimate follows from integr ability of (1 + |x|) sp_ * p if sp > s > d + 1. □ 
For the following, we remind the reader of D^g : £ 1— > g(h T ^). 

Lemma 3.5. Assume that f, if) € L 1 (M d ) /taue vanishing moments of order r in O c , and fulfill 
|/| r ,7— H < 00 1 l^lty—lal < 00 • -^ e ^ ct be a multiindex with \a\ < r. Then there exists a constant 
C > 0, independent of f and tp, such that 

\d a (f- D h M)\ < C|/lr-,r-|o ! |l^lr-,r-| Ct ||det(/z)| 1 /2 (1 + \\ h \\p A^)^ A(h T 

Proof. Using the Leibniz formula, we obtain 

7+/3=o 

For any (3 occurring in the sum, (<9^/) v has vanishing moments in O c of order r — |/3| > r — \a\. 
Hence, Lemma 13.31 yields 

^/(£)| ^ |^/lr-|«|,r-| a [^(O r - 1 ^ 1 < \f\r,r-\ a \A(O r -^ . 

On the other hand, an analogous reasoning together with the chain rule yields 

WhVXOI ^ l^lr J r-| a |(l + ll^ll) |Q| >l(^)'" _|a| • 



□ 



We now come to one of the central estimates of this paper: 
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Theorem 3.6. Let 1 < p < oo and 1 < q < oo. Assume that O is (s,q,w) -temperately embedded 
with index I. Let f^ip^ L 1 (IR d ) have vanishing moments in O c of order t>£-\-s + l + d/p. 
Then there exist constants C Ptq > 0, independent of f, ip, such that 

||wV|| l p, 9 < c p , q \f\ t Mt,t ■ 

Proof. The Haar measure on G is given by d(x,h) = dx | g^r^ji ■ Our conventions regarding the 
operator allow to rewrite 

W^f(x,h) = (idet^)! 1 / 2 /- (D^Y (x) . 
Thus, by our assumptions on the weights v, w: 



|V^/||[ P „ < / |det(fe)| 



19/2 



H 



f ■ (D h ^) 



9 (u\i dh 
w[h) q 

L? 



|det(/t)| 



By Lemma 13.41 we have 



P \a\<s+l+d/p 



d a (f-[D h ^ 



with a constant depending on p, but not on ip and /. Thus, by the L 9 -triangle inequality 

dh \ l,q 



|det(/i)| 



q/2 



a\<s+l+d/p 



d a [f-[D h ^ 



w{hf 



\dzt{h)\) 



and we will estimate the integral for each \a\ < s + 1 + d/p independently. By Lemma 13.51 we 
find 



n 
-< 



det(h)\ q / 2 \\d a (/• (D^^wihy 



dh 



|det(/t)| 
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IfltMt^detih^wmi + WhW^AiO^A^ty-^dt 



dh 



\det(h)\ 



Thus, by Minkowski's inequality for integrals, 



i-i 



det(h)\ q / 2 

=< \f\tMt,t 



d a [f-(D h i> 



i .... dh \ 1/q 



w{hf 



\det{h)\) 



-< 



R d \JH 



dh \ X l q 

\det(h)\ q / 2 w(h) q (l + ll^ll) 91 " 1 ^^)^" 1 " 1 ^^)^" 1 "^ i ; V, m , 

\det(h)\J 



and it remains to prove that the integral is finite, for all |q| < s + 1 + d/p. For this purpose 
observe that O C R d is of full measure, hence it suffices to integrate over O. We fix £o € O, 
then the stabilizer in H associated to £o is compact. Let p^ : H — > O, p^ Q {h) = /i T £o denote 
the canonical quotient map. Then the measure /i£>, defined by fJ-o(A) = hh(p^(^)) is a well- 
defined Radon measure on O, which is Lebesgue-absolutely continuous with Radon-Nikodym 
derivative 

d/z o (/i T £ ) |det(/0| 



A H (h) ' 
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see |14} [15] for more details. By the proof of [14} Theorem 13], we obtain a positive constant 
Co > such that, for any Borel- measurable F : O — >• M + , 

|det(/i)| d/t 



F(£)dt = co [ F(h T (o){^ 



ih)\ 

Applying this to the above integrand yields, after the substitution £ = hJ^Q, that 

tih \ x l q 

\det(h)\ q / 2 w(hy(l + ||fe||)gl a lA(0 9(f ~ |a|) ^(fc T O q(t " |a|) , i , \ i ^ = 
// |det(/t)|y 

J/? \ l l q 

\det(h)\^ 2 w(hy(l + ||/t||) g|al A((/tife) r ^o) g( *" |tt|) u fi\ i 



CO 



a|) |det(/ii 

dh 



|det(/if 1 /i)| 9/2 iw(7ii 1 7i) 9 (l + ||/ir 1 ^ll) 9|Q!| ^(W T Co) 9( *" |Q|) 



(det^ 1 /!)! 



1/9 



4 /lT, Nt-|a| |det(fei)| 

l\H\tll) 

where we used left-invariance of fin- Now applying the submultiplicativity (possibly up to 
constant factors) of |det|,ui and || • || yields 



r< J (J |det(/i)|«/ 2 «;(/i)«(l+||7»||)«l a lA((/ l ) r $o) ,( ^ |a|) 



dh 



\det{h)\ 



1/9 



U ;(/ tr 1 )|det(/, 1 )|^+i/9 (1 + II^IDl"!^^)*-^!-^ 

L\H{hl) 
dh \ 

w (/ il )idet(/ ll )r i ^- i / 2 (i + iifeiii)i a U(^ T e ) t - |a| dfei , 



// 

where the last equation is obtained by pulling the inner integral -which is now independent of 
hi— out, and inverting the integration variable hi, cancelling Ah (hi) in the process. Now the 
facts that t — \a\ > t — s — d/p — 1 > £, A < 1 and \a\ < s + d + 1 imply that the integrals 
are finite whenever O is (s, g, u;)-temperately embedded with index The proof is therefore 
finished. □ 



4 Vanishing moment conditions and norm estimates for coorbit 
spaces 

We can now easily derive sufficient criteria for the existence of wavelets with nice properties, 
and also for functions to be in a coorbit space. For the explicit construction of nice wavelets, the 
following lemma is quite useful, which relies on the condition that O c is a finite union of linear 
subspaces. It is important to note that this additional assumption is not particularly restrictive, 
in fact I am currently not aware of a single example of a dilation group acting irreducibly and 
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square-integrably which does not fulfil it. In dimension two, it holds for all admissible dilation 
groups (see next section). 

Lemma 4.1. Assume that O c is the finite union of linear subspaces. Then there exists a linear 
partial differential operator D with constant coefficients and degree k > with the following 
property: If f and all its partial derivates of order < sk is in L 1 (M. d ), then D s f has vanishing 
moments of order s in O c . 

Proof. By assumption, O c = Uj=i^j> anci each Vj C I" is a (necessarily proper) subset. For 
each j, pick a basis x\j, . . . , x Sji jof the orthogonal complement Vj (understood with respect to 
the Euclidean scalar product on R d ), and let d Xi . denote the associated directional derivative. 
Then, letting Lj = Y^hLi @xi ■ one readily verifies that 

D = L\ o L 2 • • • ° Lm 

acts on the Fourier side via 

(Df) A (o = p(o/(o , 

for all / € L 1 (R) with sufficiently many integrable partial derivatives. Here P is a polynomial 
vanishing precisely on O c . But then the product rule implies that D is as desired. □ 

We can now easily state sufficient conditions for admissibility: 

Corollary 4.2. Assume that O is (0, 2, uu) -temperately embedded with index £, and constant 
weight w = 1. Then any function f € L 1 (M o! ) n C i+d+1 (M. d ) with vanishing moments in O c of 
order t > I + s + d/2 + 1 and integrable partial derivatives of order up to It is admissible. If O c 
is the finite union of linear subspaces, there exist admissible vectors ip € C^°(M. d ). 

Proof. The first statement is a special case of Theorem 13.61 The existence statement follows by 
picking an arbitrary nonzero / G C™(R d ) and letting ip = D e+s+d+1 f. By construction of the 
operator D e+s+d+1 , 

ko = p(o ■ no , 

and P is a polynomial vanishing precisely on O c , with order at least I + s + d + 1. Since / is 
continuous and nonzero, / • F / 0, and thus ip is nonzero. Since in addition V^tp € L 2 (G), this 
implies that tp is admissible. □ 

The same argument, with slight adjustments, yields sufficient conditions for L 1 -admissible vec- 
tors. 

Corollary 4.3. Assume that O is (s,l,w) -temperately embedded with index L Then any func- 
tion f G L 1 (R d ) with vanishing moments in O c of order t >£ + s + d J r l and integrable partial 
derivatives of order up to 2t is in ~Hx jV . If O c is the finite union of linear subspaces, there exist 
nonzero vectors ip G C£°(R d ) n U\, v . 

We next turn to coorbit spaces. Here a direct application of Theorem 13.61 yields the following 
sufficient condition. Note that for the wavelet, we may take any element in J r ~ 1 C£°(0), which 
automatically fulfills any vanishing moment condition. 
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Corollary 4.4. Let 1 < p < oo and 1 < q < oo. Assume that O is (s,q,w) -temperately 
embedded with index £. Then any function f G L 1 (R d ) n C e+d+1 (R d ) with vanishing moments 
in O c of order t > I + s + d/p + 1 and integrable partial derivatives of order up to 2t is in 
Co(LS 9 (G)). 

For the systematic study of coorbit spaces, it is useful to introduce specific spaces of test functions 
and tempered distributions. We let 

S' 0C = {ipe S'(R d ) : supp(<^) C O c } 

and 

5(0) = {/ G S(R d ) : £ S' , : </>(/) = 0} . 

As intersection of kernels of tempered distributions, 5(0) is a closed subspace of 5(R d ), thus a 
Frechet space in its own right, containing C£°(0). Moreover, it is straightforward to check that 
the dual 5(0)' can be identified with the quotient space S'(M d )/S' c , c . This should be compared 
to the case C = {0} of the similitude group: Here S'^ is just the span of the partial derivatives 
of the 5-distribution at zero. The inverse Fourier transform is the space of polynomials, and 
it is customary to regard elements of the homogeneous Besov spaces as tempered distributions 
modulo polynomials. 

Clearly any element in J r ~ 1 S(0) has vanishing moments in C of arbitrary order. Thus, as a 
consequence of Corollary 231 we obtain the following embedding result. Note that the continuity 
statement follows from fact that the estimate in Theorem 13.61 uses the Schwartz norms. The 
density statement follows from Corollary 12.111 

Corollary 4.5. Let 1 < p < oo and 1 < q < oo. Assume that O is (s,q,w) -temperately 
embedded. Then T~ 1 S(0) C Co(hy' q (G)) is a continuous embedding, and dense for p < oo. 

As a specific consequence, we note that by duality we obtain the continuous embedding 1~Li v C 
J r ~ 1 S'(JH d )/J r ~ 1 S' C) c- This has the nice consequence that in fact all coorbit spaces (assuming the 
respective temperate embeddedness conditions) can be fitted in the following framework: We 
pick a suitable Schwartz function ip as wavelet, and can then define the wavelet transform for 
arbitrary tempered distributions by letting 

V^ip{x, h) = ((p,Tr(x,h)ip) , 

where the bracket (•, •) : S'(R d ) x S(R d ) ->• C denotes the sesqui-linear version of the duality 
between 5 and S' . This definition then extends the definition of the wavelet transform on TL\ V - 
Thus every coorbit space arises as a (quotient space of a) subspace of 5'(lR rf ). An alternative 
could consist in using the dense embedding J r ~ 1 C^°(0) C %\, v , which is available without 
any assumptions on the dual orbit. However, this space is less compatible with the Fourier 
transform; the right-hand side of the dual embedding Ti.i v C (j r ~ 1 C^°(0)) does not seem 
particularly accessible. 

5 Temperately embedded orbits in dimension 2 

In this section we will exhibit a large variety of examples for which the rather technical conditions 
of the previous sections can be verified. The following results will show that, for any admissible 
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dilation group in dimension two and a large class of weights uu (which include all weights that 
have been studied previously in this context), the dual orbit is tu-temperately embedded, which 
makes Theorem 13,61 and its various corollaries available for all these groups. Essentially, we will 
treat three different types of groups, which constitute a set of representatives of all admissible 
matrix groups in dimension 2, up to conjugacy and finite extensions. The fact that the list 
of groups treated in the following subsections is such a set of representatives is proved in [T51 
116] . Furthermore, note that the properties that we are interested in are well-behaved under 
conjugation: If H and H' are conjugate by some g G GL(K, d), then weights on H correspond 
to their conjugate images on H' , and the associated dual orbits are mapped into each other by 
g T , and thus inherit the property of being temperately embedded. Similarly, if H < H' are two 
admissible dilation group, then H has finite index in H', with the same dual open orbit |16| . 
and all relevant properties are transferred from H to H' and vice versa. One such property is 
that the complement of the dual orbit is the finite union of linear subspaces. In the following, 
three cases will occur: Either O c is just the origin, or it is a line through the origin, or the union 
of two such lines. 



5.1 The similitude group 

Let 

H = | ( M : a, b G R, a 2 + b 2 > 

This is the similitude group in two dimensions, and it is the first systematically studied higher- 
dimensional dilation group, [251 [I]. The coorbit spaces associated to the isotropic weights on H 
that we shall employ below are just the usual (isotropic) homogeneous Besov spaces. 

The dual orbit is computed as O = R 2 \ {0}. Let h = ( _° 6 M G H be given. We fix 

£ = (1, 0) T G O, then h T £ = (a, b). We assume that the weight w on H fulfills w(h) < {a 2 +b 2 ) u 

1 /2 

with u > 0. For the matrix norm, the choice \\h\\ = (a 2 + 6 2 ) will be most convenient, and 

we will use the euclidean norm on R 2 . Haar measure on H is given by ■ 

Given any £ ^ (0,0) T , one readily verifies that dist(£, C ) = |£|, and that do(0 = 0, hence 



A(0 = min |e| 



Thus, in order to verify condition (i) of Definition 13.11 we need to prove finiteness of 



f ( a 2 +6 2 )g /2-l (1 + (a 2 +6 2 ) l/2 ) (s + 3 )g(a 2 +6 2 rg min f (fl 2 + ^1/2^ . 



J V q dadb 



1 + (a 2 + 6 2 )V2 J a 2 + b 2 ' 
for I sufficiently large. But this follows easily from the observation that 

and the integrability of (1 + (a 2 + b 2 ) l l 2 )~ r whenever r > 2. 
For the second condition, we first compute 



AQT 1 £o) = min (a z + b 



l + (a 2 + 6 2 )-V2 
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hence we need to verify finiteness of 

/ (a^ + b^-\lHa 2 +b^y + Ha 2 +br min ((a 2 + 1 , ' 1/2 Y' • 

7r2\{0} V l + (a 2 + 6 2 ) V*J a 2 + b 2 

This follows by a similar argument. 

Remark 5.1. The calculations carried out for d = 2 immediately generalize to higher dimen- 
sions. 

Remark 5.2. Note that in order to apply Corollary \4-3\ to obtain analyzing vectors for the 
coorbit spaces, we need sufficient conditions for 1^ AG), where vq is a controlling weight for Ly' q . 
As the proof of Lemma \2.3\ shows, one can construct controlling weights such that the weight wq 
on H is obtained by multiplying w with certain powers of (l + Wh^ 1 ]]) and (1 + Aq(0, h^ 1 )). Each 
of these factors is controlled by suitable powers of the auxiliary function A, and thus Corollary 
\4-3\ becomes applicable. 



5.2 The diagonal group 

Let 

H = | (® M : a,b£R,ab^0 
Haar measure on this dilation group is given by t^t- The dual orbit is determined as 

O = M 2 \ ({0} x MUM x {0}) . 

For ^ = ^q ^^^^ ano ~ £° = (1'1) T e ®-> we obtain /i T £o = (a,b). We assume that the 

weight w on H fulfills w{h) < (\a\ + |a| -1 )*(|6| + H" 1 )", with u > 0. Most other reasonable 
weights (isotropic or anisotropic) can be majorized by this weight, for suitably large u. For the 
matrix norm, we choose = max(|a|, |6|); and for the norm on M 2 we use the £ 1 -norm. The 
determinant of h is ab. 

For any £ = (£i)6) T £ O, one computes that dist(£,0 c ) = min(|£|i, ^2), and <io(£) = 
max(|£i|, |f 2 |), hence 



A(£) = min 



min([£i|,|6|) 1 



,l+max(|ei|, |6l)' 1 + 161 + 161 
Thus, condition (i) of Definition 13.11 requires finiteness of 

l b\i/ 2 - 1 (l + max(|a|, |&|) ( * +3)ff (|a| + lap 1 ) 9 ^^! + Ib]- 1 )^ 
min(|a|,|6|) 1 \ £q dadb 



x mm , , , , ■ , 1 . 

1 + max(|a|, l + |a| + |6|/ \ab\ 

for I sufficiently large. In order to prove this, we first observe that 

±1 . / min(|o|,|6|) 1 \ 

|a| mm Vl + max(|a|,|6|)' \ + \a\ + \b\) ~ 
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holds, and likewise for b. Thus the product 

H*/~(i + ^m, | 6 |)<.«> 3( h + H- l) , ( | 6 | + lbrr min TT ^i TW ) ' 

is bounded for r sufficiently large, and integrable at least for r + 3. 
For condition (ii) of Definition 13.11 we need to ensure finiteness of 

ib\- l ' 2 - l l q {l + max(|o|, |fe|) s+3 (|a| + M -1 )*^! + H" 1 )' 
mmdol -1 ,^! -1 ) 1 V 9 dadb 



mm 



l + maxdal- 1 ,^]- 1 )' 1+ (a)- 1 + Ife^ 1 / |a&| 



Noting that 



min(|a 



1 + maxda]- 1 , H" 1 ) max(|a|, |6|) 1 + maxO^ 1 , \b\- r ) 
we can again estimate 

I .±1 mindal-Mftl- 1 ) 1 

II l + maxdal-Wbl- 1 ) 1 1 + la]- 1 + ^1^ ~ ' 

and likewise for b. Hence we can choose an exponent I > sufficiently large such that the inte- 
grand becomes bounded, and increasing the exponent by three results in a convergent integral. 

Remark 5.3. Also for the diagonal group, the reasoning easily extends to arbitrary dimensions. 
Again, the proof showed that functions of the type (1 + \\h ||) and (1 + Ag(0, h^ 1 )) can be 
controlled by suitable powers of the auxiliary function A. Hence, the same reasoning as for 
the diagonal group allows to conclude that Corollary \4-3\ yields sufficient criteria for analyzing 
vectors. 



5.3 The shear let groups 

Fix a real parameter c, and let 

Hc= {( a c ) : a > feGM > a ^ } • 

Here we use a c = sign(a)|a| c for a < 0. Note that for c = 1/2, this is the shearlet group 
introduced in [23], and studied (e.g.) in [6j C] . The other groups are obviously closely related; 
the parameter c can be understood as controlling the anisotropy used in the scaling. Haar 
measure on this group is given by db ^ +c . The dual orbit is computed as 

O = M 2 \ ({0} x M)) . 

For h = f q ^ c ^j S H and £o = (1;0) T G O, we obtain /i T £o = ( a ,b). We assume that the 

weight w on H fulfills w(h) < (|o| + | cx | 1 + \b\) u . This is the weight employed in [6]. For the 
matrix norm, we choose \\h\\ = max(|a|, \a\ c , \b\); and for the norm on M 2 we again employ the 
£ 1 -norm. The determinant of h is a 1+c . 
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For £ G O, we have dist(£, O c ) = |£i|, and we find do(£) = l&l- Hence condition (i) in Definition 
13.11 requires finiteness of 



|(-«//2-i/)(i+c)( 1 + max (| a | ) | a | Cj \b\))( s +Vi(\ a \ + (ar 1 + |6|) 



uq 



mm — -, — ; — - db-, 



l + l&l' 1 + H + l«l 2+c 

for sufficiently large £. This is easily achieved by the balancing argument used for the previous 
cases, since 

max(|a|>r\H) • min (J^L, , - i - - ) < 1 , 
and (1 + |a| + |6|)~ 2 integrable. 

For the second condition, we first compute = I ^„ c j . Thus condition (ii) 

translates to 

[ [ lalC-V^-i/^Ci+c)^ + max (| a | 5 |5|))3(| a | + | a |-i + | 6 |^ 

JR' JR 

( \ a V X 1 \ l ii da 

x mm — r— -, — -— ; — -r-— do— ■= < oo . 

Vl + lal-"- 1 !^' 1 + \a\~ l + |ctj— c — 1 |fo| y \a\ 2 +c 

To see that this holds for large enough £, we first note that 

I i±i • ( \^ 1 
a mm 



lal^ 1 min 



1 + — ^ — x | fo| ' 1 + |a-|— 1 + | ct | — c — 1 |fo| 

/ \ 

1 \a\ 
\a\ + |a|- c |6|' 1 + \a\ + |a|" c |6| 



< 1 • (11) 



V < |a.| — 1 <]a| / 

In addition, since 

\b\ = , |c \b\ < , |c 
\a\ + |a" c |6| |a ' |a| 1+c + \b\ ~ |a| ' 

inequality (fTTI) implies that 



l&l m i n I : — i 7777 , : — ; — : : — ; 77-r < 1. 

11 \1+ \a\~ c - 1 \b\ 1 + \a\~ 1 + \a\~ c - 1 \b\J 
Thus the usual course of argument, together with 

min ( r^r 77-7 , r-p-i th 7T7T ) dbda 

1 + — c — 1 1&| 1 + la]- 1 + \a\- c - l \b\) 

m i n In rn — m ^ i i i — ml dbda 

\\a\ + |a|- c |6| l + |a| + |a|- c |6|y 

< 4 / / C — -1 dbda + 4 / / f dbda 

~ Ji Jo \a + a~ c bj J J \l + a + a- c bj 

< oo , 
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as soon as r > max(2, c + 2), yields finiteness of the integral for sufficiently large £, and we have 
established that the orbit is w-temperately embedded. 

Remark 5.4. The calculations show that the index £ may depend on the parameter. Hence, even 
though the dual orbits coincide for all groups, the indices involved in temperate embeddedness 
(and thus: the number of vanishing moments required for the weighted L p ' q -estimates) may 
depend on the group. This serves as a reminder that temperate embeddedness is not strictly a 
property of the orbit (as a set), but rather a property of the group and the way it acts on the 
orbit. 

Remark 5.5. The same reasoning as in the previous two cases allows to verify that Corollary 
\4-3\ yields sufficient criteria for analyzing vectors. 

Remark 5.6. As already mentioned, the case c = 1/2 corresponds to the shearlet group. Thus 
the results of Section [^] all apply to this group. Many results in this paper have shearlet coun- 
terparts in the literature (e.g. in J^\^) that typically are similar in spirit but do not exactly 
match in terms of the required assumptions. As examples we mention Theorem \2.1\ Lemma \2. 1\ 
Corollary \2.11\ Corollary \4-3\ I am not aware of a shearlet version of Corollary \4-4\ 
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